Validity and Reliability Reports in Applied Linguistics Research Articles: The case of tests and questionnaire

This study intended to determine the way validity and reliability i.e., psychometric properties were reported in the Applied Linguistics research articles. The study also focused on the measurement methods applied to determine the validity and reliability of the scores derived from the tests and questionnaires in the empirical studies. The corpus of the study included 331 empirical studies derived from 733 research articles (RAs) published between 2005 and 2018 in three prominent Applied Linguistics journals – Applied Linguistics, Modern Language Journal, and TESOL Quarterly, The selected papers used test and/or questionnaire for data collection. Our analysis indicated that 77(20.98%) of the studies did not report validity and reliability measures, 82(22.35%) reported only reliability measures, 26(7.08%) reported only validity measures, and 182(49.59%) reported both the validity and reliability measures for the instruments. It was also found that content validity assessed through the pilot study had the highest frequency among validity evidences while internal consistency, mostly identified by Cronbach's alpha, was the most frequent reliability evidence.